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Data is easy. Deciding is hard. 



The data-to-decision 
process is broken. 

Data isn't being delivered to decision makers the way they want. And they're part of the 
problem. 

Here's what our research says about barriers getting in the way. The entire data-to-decision 
process needs to be computable: From data, to analysis, to action, to outcome. 

To be recognized as peers and business partners, analysts need to connect actions with 
outcomes, and do more to manage the decision process. Meanwhile, decision makers should 
strengthen analytical skills and explain their rationale. For this to happen, we need to resolve 
these six issues: 

Data is easy: 3 issues for the tech community. 

1 . Too much 'Ooh, shiny! ' 

2. False dichotomy: Data-driven vs. human intuition. 

3. Value is not self-evident: Show people how to use data. 

Deciding is hard: 3 issues for decision makers. 

1. Visualize decisions, not data. 

2. Build a decision culture, not a data culture. 

3. Unleash data's explanatory and predictive power. 
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Data is easy. Deciding is hard. 



3 issues for the tech community. 

The challenge: Create engaging processes 
that show people how to use data, and connect 
actions with outcomes. 

Tech issue #1. Too much 'Ooh, shiny!' 

Machine learning, predictive analytics, tech 
wearables, and industrial Internet of Things. 
Visualizations help analysts tell their data story, 
so decision makers quickly see relationships 
and absorb complex information. 



''Chock them so full of 
facts they feel stuffed, 
but absolutely brilliant 
with information. Don't 
give them any slippery 
stuff like philosophy or 
sociology/' 

-Ray Bradbury, Fahrenheit 451 



Form over substance. But some would have you believe cool charts are the end game. In 
that respect, data is too easy (Altman). Consider today's fashionable data visualizations, and 
the dreaded infographic. Plus the statement by the CEO of Tableau, speaking to the New 
York Times, that visualizations should include "No more than 18 colors at once. You can't 
consume more than 18." 




Sources: Edward Tufte, Flowing Data, Wikipedia 



BI gets bigger, not better. Business intelligence usage is flat. A popular 2014 survey 
reported a 6% decline in those finding significant impact, down to only 28% (Swoyer of 
TDWI). Frank Bien tells the hard truth: "The common view of the past five years is that 
users are stupid and that data needs to be spoon-fed to them via pretty pictures. . .. It's time to 
strike a new balance: to join 'big data' to business data in such a way that it serves the 
business - and doesn't just grow a big data repository." 
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Data is easy. Deciding is hard. 



Not everything is an actionable insight. It's trendy to 
describe dashboards or statistic as insights. Indeed, 
some of them are. Yet boil-the-ocean projects typically 
get in the way of achieving a specific purpose. 
Bertolucci reminds us that "Just saying 'insight' and 
'innovation' is a wonderful thing, but first and foremost 
you need to focus". Tyler Vigen's Spurious Correlations 
project illustrates how pointless analytics can be (e.g., 
association between drowning and Nicholas Cage). 

What insights would really benefit decision makers? Hard evidence showing which actions 
can influence results. Plain-English explanations of useful decision methods. Prediction of 
what's will happen if they choose door number one, two, or three. 



Tech issue #2. False dichotomy: Data-driven vs. human intuition. 

Members of the tech community sound inexperienced when suggesting that, until now, 
people's decisions were based solely on intuition. As if they're rescuing us from the 
information dark ages and ushering in a time of data-driven enlightenment. 

Smart decision-making is more complicated than becoming 'data-driven', whatever that 
means exactly. Consider the challenge of augmenting the performance of a highly skilled 
professional. Investor Vinod Khosla claims technology will replace 80%+ of physicians' 
role in the decision-making process. "Human judgment simply cannot compete against 
machine-learning systems that derive predictions from millions of data points" (quoted by 
Stephanie Lee). Perhaps so, but it's tricky to blend evidence into patient care processes: 
Karen Nanji describes mixed results from clinical decision support technology. 

One tech enthusiast compares 
IBM's Watson to a hospital 
CEO. Ron Shinkman asks if it 
could "be programmed to pore 
over business cases, news 
clippings, algorithms and 
spreadsheets to make the same 
recommendations?" Actually, 
that's what Watson does. But 
Shinkman overlooks the real 
opportunity: To supplement, 
not replace, a CEO's 
analytical skills. 



It s a mistake to 
describe analytics 
adoption as a battle 
between two mythical 
beasts: Data Driven 
and Human Intuition. 



Intelligent enterprises will adopt new ways of working. 



Traditional approach 



Instinct and intuition 



New approach 



Fact-driven 



Corrective 



Years, montlis, weeks 



Hours, minutes, seconds ' 



Human Insigtit 



Applied semantics 



Decision support 



Action support 



Optimized 



Source: Business analytics and optimization for the 
intelligent enterprise (IBM). 
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Data is easy. Deciding is hard. 



Why IT Fumbles Analytics. In an excellent Harvard Business Review analysis of how 
decision makers assimilate data, Marchand and Peppard explain that management lacks 
"structure. Even when an organization tries to capture their information needs, it can take 
only a snapshot, which in no way reflects the messiness of their jobs. At one moment a 
manager will need data to support a specific, bounded decision; at another he'll be looking 
for patterns that suggest new business opportunities or reveal problems." 

You're not the boss of me. There's a right 
time and a wrong time to look at data. As 
Peter Kafka explains, Netflix executives 
enthusiastically use data to market TV 
shows, but not to create them. Others agree 
data can interrupt the creative process. In 
The United States of Metrics, Bruce Fieler 
observes that data is often presented as if it 
contains all the answers. But "metrics rob 
individuals of the sense that they can 
choose their own path." 

However, people could do better. Of 

course decision makers frequently should 
ignore their instincts. Andrew McAfee 
gives examples of algorithms that 
outperform human experts, and explains 
why our intuition is uneven (we need cues 
and rapid feedback). 

The Economist Intelligence Unit asked 
managers "When taking a decision, if the 
available data contradicted your gut feeling, what would you do?" Most preferred to crunch 
some more numbers. Only 10% said they would follow the action suggested. The sponsors 
of Decisive action: How businesses make decisions and how they could do it better 
concluded that while "many business leaders know they need to make better use of data, it's 
clear that they don't always know how best to do so, or which data they should select from 
the enormous quantity available to them. They are constrained by their ability to analyse 
data, rather than their access to it." 

How do you challenge a decision maker? When data is available to improve a result, it 
must be communicated so it challenges people to apply it. One way is to provide initial 
recommendations, and then require anyone who takes exception to enter notes explaining 
their rationale. Examples: Extending offers to customers on the telephone, or prescribing 
medical treatments. 



Jul 23, 2014 9:55 AM 

Data: Q3 product reveni 
missed forecast by 4.59 



4 9:55 AM 

snue ^ 

5% ] 



Jul 23, 2 014 9:55 AM 

Human: You're not the 
Boss of me. 

Jul 23, 2014 9:55 



Data: Wait. What!!? 



Jul 23, 2014 9:55 

Human: Let's work 
together. 

Q [leyX Message 
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Data is easy. Deciding is hard. 



Tech issue #3. Value is not self-evident: Show people how to use data. 

A compelling presentation to decision makers will propose and answer these questions: How 
are the findings connected to identified business objectives! What analytical methods were 
used? Who can use these findings, and where are they applicable! 

Show why it matters. Accenture's extensive research has found that "most organizations 
measure too many things that don't matter, and don't put sufficient focus on those things that 
do, establishing a large set of metrics, but often lacking a causal mapping of the key drivers 
of their business." 

SAP's chief data scientist emphasizes the importance of someone "who can translate PhD 
to English. Those are the hardest people to find" (as told to James Willhite in Getting started 
with big data). Kerem Tomak, who manages 35 retail analysts, has learned "A common 
weakness with data analytics candidates is they're happy with just getting the answer, but 
don't communicate it" (speaking with Shane O'Neill of Information Week). 

Tech designed for decision makers. Shown below is one way to demonstrate the value and 
use of data. A specific decision is visualized, explicitly connecting an action with an 
outcome. This causal connection is explained with data. 



Demonstrate Value: Connect Actions & Outcomes 



Show the 
decision ""^^^^^ ^ 



Explain 
with data 



If you take 
this action 



Here's why: 



^1 Evidence 

Method: Linear 
Source: RCT 
"Historical trends 
show that ..." 



You can 
expect this 
outcome. 




mDataSci 

Method: Bubble 
Source: Customer 
"Analytics predict 
better outcome..." 



II 



Cruncher 

Method: Dynamo 
Source: SAS 
"Evidence shows 
this action will..." 



Source: Ugly Research. 



©2014 Ugly Research 



5 of 11 



Data is easy. Deciding is hard. 



3 issues for decision makers. 

As decision makers, we've been slow to break 
bad habits: We don't transparently show our 
rationale. We don't hold people accountable for 
their decision processes. We don't do enough to 
strengthen our analytical capabilities, and rely too 
heavily on expert advisers. 



'There needs to be 
a user interface 
between the data and 
the decision maker/' 

-Sean Gourley. CTO Quid 



Decision issue #1. Visualize decisions, not data. 

Decision-making would be less painful if we could more easily see what the decisions are. 
In Analytics in action, Accenture underscores the "need to industrialize the insight-action- 
outcome sequence". Instead of sales forecasts, visualize sales management, connecting 
specific actions and outcomes along a continuous data-to-decision process. 



We Have Met the Enemy, and 
He Is PowerPoint 
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Source: New York Times. 



Better than a persuasive guy with a 
PowerPoint. Tom Davenport provides a 
great example in How P&G presents data to 
decision-makers. With Tibco Spotfire, P&G 
developed a common language and created 
seven models thath guide the analysis of 
various problem domains. These include 
"key variables, how they should be 
displayed visually, and (in some cases) the 
relationships between the variables and 
forecasts." 

Ferraris vs. geopolitics. Of course many 
problems can't be easily diagrammed. 
Wladawsky-Berger reminds us of the 
difference between figuring out a Ferrari 
(predictable) and geopolitics (not so much). 
Applying "big data and data science to help 
with strategic decisions is in its early stages." 



Snowden and Boone explain that "Simple 
and complicated contexts assume an ordered universe, where cause-and-effect relationships 
are perceptible, and right answers can be determined based on the facts. Complex and 
chaotic contexts are unordered - there is no immediate relationship between cause and 
effect, and the way forward is determined based on emerging patterns." 
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Data is easy. Deciding is hard. 



A repeatable, visual decision process 
requires transparency. 

Be brave. Transparency requires a culture shift. 
Barriers include people's nervousness about 
scrutiny of their methods, the difficulty 
of capturing complex analysis, and concerns 
about proprietary knowledge. 

The National Pharmaceutical Council led a recent 
effort to shed light on how health insurers make 
formulary decisions. As explained by Dean et al., 
investigators found it next to impossible to 
systematically capture a complex decision rationale 

Close the loop. Disconnected processes pose another challenge, even when individual steps 
are well executed. Example: In private conversations with analysts at a multi-billion dollar 
energy producer, it's clear they use sophisticated methods for valuing potential investments. 
However, they are frustrated that once an annual list of candidates is developed, there's no 
consistent portfolio analysis for making company-wide decisions, and instead too much 
reliance on the persuasive guy with a PowerPoint. 



Decision issue #2. Shift from a data culture to a decision culture. 

Shift focus from data to decisions. James Taylor of Decision Management Solutions 
suggests beginning with the decision in mind, "identifying the decisions that matter to your 
organization, the decisions that make the difference between hitting your targets and missing 
them, the decisions that 'move the dial. . ..' It means beginning with a model of decision- 
making and only then moving on to identify the analytics and data, big or little, that will be 
required." 

Process is king. In a decision culture, people are expected to collaborate on repeatable 
processes, applying the right analytical method to each decision. A bad result doesn't 
necessarily reflect a bad process - or a bad decision maker. In Are analytics shifting power 
from executives to employees?, Gary Cokins describes how decisions are being redistributed 
within the organization: More people have more data, and are involved in more decisions. So 
there has to be a process. 

Innovating for decision makers. How is technology helping people? Data scientists are 
setting up labs where business managers can sharpen analytical skills. Computerized 
Provider Order Entry systems embed logic and medical evidence to improve healthcare 
decisions. Predictive analytics tools make it easier to explore possible outcomes. And 
prescriptive analytics technology recommends actions (such as replacing manufacturing 
equipment), tracking and reporting actual outcomes to close the loop. 



''Companies fail to 
embed analytical insights 
in key decision processes 
so that analytics 
capabilities are linked to 
business outcomes/' 

-Accenture, Analytics in Action 
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Data is easy. Deciding is hard. 



Decision issue #3. Unleash the power. 

Data's explanatory and predictive power can 
help us make better decisions. But explanations 
and predictions aren't being managed rigorously 
as part of the data-to-decision process. 

Explaining is tricky. To explain is to describe 
facts and their consequences, linking action with 
outcome. All this gets deeply philosophical - see 
Cornwell's work on scientific explanation. 



Decision makers need 
practical explanations that 
are structured pieces of a 
data-to-decision process, 
not one-off visualizations. 



David Stodder of TDWI finds that only 45% of applications for visual analysis include 
'embedded explanations' - but these are typically for bounded decisions. We're doing too 
much reporting, and not enough explaining, as shown below. We need to solve the problem 
of embedding explanations of big, unbounded, managerial decisions. 



What's Next? Structuring Explanations and Predictions 



r 



unbounded- 



cu 
Q. 

C 

o 

'u 
cu 
Q 



bounded- 



automated- 



Report 
outcomes. 



data 



Figure out 
improve outcomes 



It how to f 



Explain & predict 
outcomes. 




analytics decision management 

Information type 



Source: Ugly Research. 
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Data is easy. Deciding is hard. 



Prediction: Bright future ahead. To predict is to claim a certain outcome will be the 
consequence of a certain action. Predictive analytics make evidence more tangible, giving 
decision makers a practical way to explore possible outcomes and achieve a comfort level 
with the actions they consider. 



To fully benefit from data's predictive power, make 
prediction part of a systematic data-to-decision process. 
Decision makers should expect to see explanations 
integrated with predictions. 

Shmueli has explored the distinction between explaining 
and predicting. They're closely related, but in research 
circles, explanation gets most of the attention. Prediction 
is often considered unscientific, and treated like a second 
class citizen. That needs to change. 



If you don't turn 
data ''into a usable 
prediction, if s hard 
to consume it/' 

-James Taylor, CEO Decision 
IVIanagement Solutions 



The Last Word 

Fixing the data-to-decision process will require us to shift our thinking, collaborate on user 
interface design, and break bad habits. To be recognized as peers and business partners, 
analysts need to connect actions with outcomes, and help manage the decision process. To 
get more value from data, decision makers need to strengthen analytical skills and 
transparently explain their rationale. 

Fancy charts are not enough. The entire data-to-decision process needs to be computable: 
From initial data, to analysis, to action, to outcome. The trick is creating something that's 
straightforward, but capable of representing complex decisions. 



©2014 Ugly Research 



9 of 11 



Data is easy. Deciding is hard. 



References. 

Accenture. Analytics in action: Breakthroughs and barriers on the journey to ROL August 
2013. 

Altman, Tracy Allison. Enough already with the ooh, shiny! data. Show me evidence to 
explain these outcomes . Evidence Soup blog. 29-Jan-2014. 

Asay, Matt. More big data is better, but good luck understanding it all. ReadWrite. 30-Jan- 
2014. 

Bertolucci, Jeff. Big data fans: Don't boil the ocean . Information Week. 12-May-2014. 

Bien, Frank. 3 bad business-intelligence habits that we need to break . VentureBeat, 7-May- 
2014. 

Bumiller, Elisabeth. We have met the enemy and he is PowerPoint. New York Times. 26- 
Apr-2010. 

Cartwright, Nancy. Hunting Causes and Using Them. Cambridge U Press. 2007. 

Cokins, Gary. Are analytics shifting power from executives to employees? Information 
Management, 6-Feb-2014. 

Cornwell, John (Ed). Explanations: Styles of Explanation in Science. Oxford. 2004. 

Davenport, Tom. Looking outward with big data: A Q&A with Tom Davenport , Strategy + 
Business. 31-Mar-2014. 

Dean, Bonnie B, et al. Transparency in evidence evaluation and formulary decision-making . 
Pharmacy and Therapeutics, Vol 38, no 8, August 2013. 

Economist Intelligence Unit. Decisive action: How businesses make decisions and how they 
could do it better . 5-June-2014. 

Fieler, Bruce. The United States of metrics . New York Times, 16-May-2014. 

Gourley, Sean. Quid CTO, DataBeat conference. 5-Dec-2013. 

Hardy, Quentin. A makeover for maps . New York Times. 6-Jan-2014. 

IBM, Business analytics and optimization for the intelligent enterprise . 2009. 

Kafka, Peter. Netflix loves big data, but won't use it to make TV shows (Video). Recode, 1- 
June-2014. 

Lee, Stephanie. Vinod Khosla: Doctors cannot compete with machines . The Tech 
Chronicles. 23-May-2014. 

Marchand, Donald A. and Peppard, Joe. Why IT fumbles analytics . Harvard Business 
Review Magazine. Jan-Feb 2013. 

McAfee, Andrew. The future of decision making: Less intuition, more evidence . HBR Blog 
Network. 7-Jan-2010. 



©2014 Ugly Research 



10 of 11 



Data is easy. Deciding is hard. 



McStravick, Alan. Predictive analytics stepping front & center in the business world , Silicon 
Angle. 2-May-2014. 

Nanji, Karen C, et al. Overrides of medication-related clinical decision support alerts in 
outpatients . Journal of the American Medical Informatics Association, 28-Oct-2013. 

Novet, Jordan. What big-data VCs are sick of - and what they really want . Strata 
Conference. 16-Feb-2014. 

Olavsrud, Thor. Even data-driven businesses should cultivate intuition , CIO.com. 7-June- 
2014. 

O'Neill, Shane. Data scientists want respect and results . Information Week. 14-Apr-2014. 

Ross, Jeanne et al. You may not need big data after all . Harvard Business Review Magazine. 
December 2013. 

Shinkman, Ron. Could Watson replace high-paid hospital execs? . Fierce Health Finance. 
Accessed 21-April-2014. 

Shmueli, Galit. To explain or to predict? . Statistical Science, v25 n3, op. 289-310. 2010. 

Snowden, David J. and Boone, Mary E. A leader's framework for decision making . Harvard 
Business Review Magazine. November 2007. 

Spence, Robert. Information visualization: Design for interaction . Prentice Hall. 2007. 

Stodder, David. Data visualization and discovery for better business decisions: TDWI best 
practices report , TDWI. Q3 2013. 

Swoyer, Stephen. BI Use in Holding Pattern. TDWI, 6-May-2014. 

Taylor, James. To succeed with big data, begin with the decision in mind . Information 
Management. lO-Feb-2014. 

Tita, Bob. Corporate economists are hot again . Wall Street Journal. 7-March-2014. 

Vigen, Tyler. Spurious correlations . Site visited 15-May-2014. 

Willhite, James. Getting started in big data . Wall Street Journal. 4-Feb-2014. 

Wladawsky-Berger, Irving. Data-driven decision making: promises and limits , CIO Journal. 
27-Sept-2013. 

Wladawsky-Berger, Irving. Data science: From half-baked ideas to data-driven insights . 
Wall Street Journal. ll-Apr-2014. 



©2014 Ugly Research 



11 of 11 



€9 UGLYRESEARCH 



About Ugly Research. 

Ugly Research is an analytics technology company based in Oakland, California. We 
connect decision makers with people who have actionable data. Our PepperSlice app 
makes it happen: Think of it as matchmaking for analytics. To find out more, contact us 
at 510.891.1033, @UglyResearch, or ur@uglyresearch.com. 

Tracy Allison Altman, PhD is the founder of Ugly Research. After years in management 
roles, Tracy was dissatisfied with the available methods for finding data that could 
explain or improve outcomes. So she created something that worked better, and 
formalized it as PepperSlice. Find her on Twitter @EvidenceSoup. 



